Troubleshooting the WEM

 

This appendix provides information on troubleshooting the following:

•	Issues Pertaining to Installation

•	Issues Related to Starting WEM

•	Issues Related to Login

•	Issues Related to the Web Browser

•	Issues Pertaining to CORBA Communication

•	Issues Related to Bulkstatistics

•	Issues Pertaining to Configuration Backup

•	Issues Pertaining to Alarms

•	Issues Pertaining to the Process Monitor (PSMON)

•	Issues Pertaining to Starting and Stopping EMS Processes

•	Issues Pertaining to Java

•	Issues Pertaining to WEM Upgrade

In addition to the above, instructions are also provided for capturing client and server logs. These are provided in the Capturing WEM Client Logs and Capturing WEM Server Logs using Script sections of this appendix.

Important: Unless otherwise specified, all information provided in this chapter applies to both Solaris and Red Hat Enterprise Linux-based WEM systems.

Issues Pertaining to Installation

Problem:	Installer window doesn't appear.
Possible Cause(s):	• If you received the “ERROR: could not initialize interface awt - exception: java.lang.InternalError: Cannot connect to X11 window server using ':0.0' as the value of the DISPLAY variable.” message, the display settings of your terminal program may be incorrect, or Exceed is not running on the client machine. • The /tmp directory may be full.
Action(s):	• Verify the display settings of the terminal application on the client machine are correct. • Verify that Exceed is installed properly on the client machine. • Determine the status of the /var/tmp directory by entering the df -k command. If it is at or near capacity, choose another directory for the Host Base Directory parameter setting. This parameter can be set via the installation process.

‌

Problem:	Received “Unable to install Element Management System <version> over Element Management System: Installed product has newer version.” message when attempting installation.
Possible Cause(s):	• A later version of the WEM is already installed. • A previously installed version of the WEM was not completely uninstalled, or was uninstalled incorrectly.
Action(s):	• Completely uninstall the newer version and install the desired version. • Determine if WEM packages exist in the /var/sadm/pkg directory. These packages begin with “EMS”. If packages exist, remove them by entering the pkgrm -n EMS* command. Once they’ve been removed, reinstall the application.

‌

Problem:	WEM components did not start successfully upon installation.
Possible Cause(s):	A previously installed version of the WEM was not completely uninstalled, or was uninstalled incorrectly.
Action(s):	• Check the installation log files (located in the directory) and EMS process log files (located in the <ems_dir>/server/log directory). • Enter the ps -ef | grep server command to determine if any process instances from previous installations are running. If so, stop them using the instructions in the WEM Server Files and Operation chapter of this guide. Once stopped, start the processes for the current installation using the instructions in the same appendix.

‌

Issues Related to Starting WEM

Problem:	WEM server doesn't start.
Possible Cause(s):	• The PostgreSQL database is not running. • The log directory may have been accidentally deleted. • Processes from a previous WEM are still running. • The “ServerPort” and/or “ServerIIOPPort” port values configured for the WEM are in use by other processes. • The physical IP address of the WEM server might have been changed without modifying other related WEM configuration appropriately.
Action(s):	• Verify that the Postgres database is running by entering the ps -ef | grep post command. If is not, follow the instructions in WEM Server Files and Operation chapter of this guide to start it. • Determine if the log directory exists in <ems_dir>/server (default directory) using the ls command to display the contents of the directory. If it is missing, create it using the mkdir command and stop and restart all WEM processes using the instructions in WEM Server Files and Operation chapter of this guide. • Enter the ps -ef | egrep “server|bulkstatparser|bulkstatserver|scriptsrv” command to determine if WEM server processes are running and, if they are, what directory did they originate from. If they’re different, stop the processes and restart the server from within the desired installation directory using the instructions in WEM Server Files and Operation chapter of this guide. • Determine if the “ServerPort” and/or “ServerIIOPPort” port numbers specified in the nms.cfg file (located in the <ems_dir>/server/etc directory by default) are already in use. The default “ServerPort” is 22222, and the default “ServerIIOPPort” is 15000. This can be determined by entering the netstat -a command which displays a list of all the process addresses and ports in use in “ipaddress.port” format. If they are in use, either stop the other processes or configure new values for these parameters. • If the WEM server IP address has been changed, the appropriate modifications need to be made in the nms.cfg and /etc/hosts files, as per the WEM IP Address Change Procedure .

‌

Problem:	Postgres doesn't start.
Possible Cause(s):	• A .s.PGSQL.5432.lock lock file is present in the /tmp directory prior to starting postgres. • Shared resources are not released after another Postgres instance was terminated. • The PostgreSQL system environment variables were not configured properly prior to installation.
Action(s):	• If the lock file is present, delete it using the rm .s.PGSQL.5432.lock command. • Determine if a previous Postgres instance is still using system resources by entering the ipcs command. If it is, clear the resources by entering the ipcrm command. • Ensure that the PostgreSQL system environment variables were configured properly using the information in Installing the WEM Software chapter of this guide.

‌

Problem:	Received the following error message in postgres log: “2008-09-13 15:57:31 IST bsdb ERROR: column "data_touseravg_bps" does not exist at character 4332008-09-13 15:57:31 IST bsdb STATEMENT: SELECT "sampledate", "sampletime", "vpnname", "vpnid", "apn", "acc_req_sent", ... FROM apn_current WHERE ("boxername"='bngnc22') AND ( ( ("sampledate"='2008-09-13') AND ("sampletime">='33900') ) ... ORDER BY "sampledate", "sampletime", "vpnname", ...; ” NOTE: The table name in the above message is after the 'FROM' keyword.
Possible Cause(s):	The mentioned table in error logs may not be in sync with the old table. There are two tables for each bulkstat subsystem in database: old and current (for example, card and card_current, port and port_current, etc.).
Action(s):	There is one workaround for this but there will be data loss for that mentioned counter. Manually drop the column from table to make the tables in sync. There is data loss only for that particular column. After this, restart all the bulkstat processes (bulkstatparser and bulkstatserver). After restarting, bulkstatparser will re-add that column in both the tables and data will be populated for that column.

‌

Problem:	Received the following error message in postgres log: “ERROR: trigger "xxxxxxxxxxxxxxxxxxxx" for table "yyyyyyyyyyyyy" does not exist ERROR: function ffffffffffffffffffff() does not exist ERROR: view "vvvvvvvvvvvvvvvvvvvvv" does not exist”
Possible Cause(s):	Error messages regarding function, trigger and view are logged in logfile when the DROP statement is executed while initializing the EMS databases. SQL file used for initializing the database contains a sequence of DROP and CREATE statements for every function, trigger and view. These messages do not have any impact on postgres as well as WEM functionality.
Action(s):	No action required. This is a normal behavior.

‌

Issues Related to Login

Problem:	Could not login to WEM.
Possible Cause(s):	Invalid user name or password.
Action(s):	Verify that the username and password you are entering is correct.

‌

Problem:	Received “Could not connect to server, destroying applet” message.
Possible Cause(s):	• WEM Server is not running. • Missing Interoperable Object Reference (IOR) files. • A firewall is prohibiting communication between the client and the server. • The user could have been locked out due to multiple failed login attempts.
Action(s):	• Verify that server processes are running using the information in WEM Server Files and Operation chapter of this guide. • Verify that IOR files are present; they are stored in the <ems_dir>/client/<ems-version-number>/ior directory by default. A number of files ending in .ior should be present. These files pertain to various functions supported by the WEM. • Edit the img.html file (located in the <ems_dir>/client directory (by default) to use fixed ports and open the required ports in the firewall. This requires the configuration of the “FIXED_PORT”, “FIXED_PORT_RANGE_START” and “FIXED_PORT_RANGE_END”. • Verify that all the ports between FIXED_PORT_RANGE_START and FIXED_PORT_RANGE_END (both inclusive) are open on the firewall. If they are not, they should be opened. • If the user is “superuser”, the set_superuser_password script can be used to reset the “superuser” password to the default. If the user is not “superuser” then the administrator needs to be contacted to reset the user’s password.

‌

Problem:	Received “Java policy file is outdated or missing” message.
Possible Cause(s):	The .java.policy file is either missing from the user’s home directory on the client machine or it has expired.
Action(s):	• Verify that the .java.policy file is present in your home directory. Refer to Preparing and Using the Client Workstation chapter of this guide for more information. • Copy the .java.policy file from the “Java Policy File” link provided in img.html file to your home directory. Ensure that no extension (i.e., .txt) is appended to the file. Important: All instances of the browser must be closed and restarted after the policy file has been updated.

‌

Problem:	Received “Server could not establish connection with client, therefore notifications will not work.” message.
Possible Cause(s):	• A firewall is prohibiting communication between the client and the server. • The Hosts file does not have an entry for the client PC's hostname and corresponding physical IP address.
Action(s):	• Edit the img.html file (located in the <ems_dir>/client directory (by default) to use fix ports and open the required ports in the firewall. This requires the configuration of the “FIXED_PORT”, “FIXED_PORT_RANGE_START” and “FIXED_PORT_RANGE_END”. • Verify that all the ports between FIXED_PORT_RANGE_START and FIXED_PORT_RANGE_END (both inclusive) are open on the firewall. If they are not, they should be opened.Edit the Hosts file (located in /etc directory) with the client PC's hostname and corresponding physical IP address.

‌

Problem:	Superuser (any user) account locked.
Possible Cause(s):	The configured number of consecutive failed logins has been reached.
Action(s):	Check the configuration of the no limit ConsecutiveFailLogin parameter in the ua.cfg file (located in the <ems_dir>/server/etc directory by default). If users are frequently locked out due to reaching the maximum limit, you may consider increasing the limit, or disabling the functionality. You may also consider reducing the amount of time the account is locked out by modifying the configuration of the no locked out LockOutInterval parameter also contained in the ua.cfg file.

‌

Issues Related to the Web Browser

Problem:	WEM Client cannot be started.
Possible Cause(s):	JRE 1.4.x is installed on the client machine.
Action(s):	JRE 1.4.x is not supported. If it is currently installed on the client machine, uninstall it and install either JRE 1.5 or 1.6.

‌

Problem:	Not able to invoke help
Possible Cause(s):	The browser is configured to block pop-ups.
Action(s):	Configure your browser to allow pop-ups from the WEM server.

‌

Problem:	WEM Client-Server version mismatch message is received.
Possible Cause(s):	Attempting to login into an older version of the WEM after having logged into a newer version of the application.
Action(s):	This is caused by the browser storing the.jar files for the newer version of the WEM client in its cache. • For JRE versions greater that 1.5: The Temporary Internet Files group in the General tab of the Java Control Panel should be used to disable caching. • For JRE versions greater that 1.6: The Temporary Internet Files group in the General tab of the Java Control Panel should be used to disable caching.

‌

Problem:	After WEM upgrade or installation of a new version, various screens, for example Graph/ Current Alarm View, do not open.
Possible Cause(s):	Check the java console and if you get exceptions like "Exception in thread "AWT-EventQueue-2" java.lang.NoClassDefFoundError", it could be the case that the browser cache is enabled on your workstation and needs cleanup.
Action(s):	Cleanup the temporary internet cache from your browser and re-invoke the WEM Client.

‌

Issues Pertaining to CORBA Communication

Problem:	IMG is unmanageable.
Possible Cause(s):	• The WEM Server cannot communicate with the system due to network issues. • There is an ORBEM client identification mismatch between the chassis and WEM. • The ORBEM client on the chassis is disabled. • There is an IIOP port configuration mismatch between the chassis and WEM. • The IIOP transport parameter on the chassis is not enabled.The chassis is unmanageable.
Action(s):	• Ensure ICMP connectivity between the system and the WEM Server using the ping <wem_server_ip_address> command from the chassis’ command prompt. Refer to the Command Line Interface Reference for more information on using this command. • Verify that the ORBEM client identification on the chassis matches that configured on the WEM. The configuration of this parameter on the chassis can be determined by entering the show configuration | grep client CLI command. In WEM, check the ASID (Application Server ID), Port, and SSL-enabled flag (IIOP/SIOP) on the Modify IMG screen. Change these settings as needed. • Check the status of the ORBEM client on the chassis by executing the show orbem client id <client_id> command on the chassis. The “State” should be “Enabled”. If the “State” is “Disabled”, execute the activate client id <client_id> command in the ORBEM Configuration Mode and check the status again-- it should now be “Enabled”. • Verify that the configuration of the IIOP port on the chassis matches that configured for the WEM. The configuration of this parameter on the chassis can be determined by entering the show configuration | grep iiop-port. In WEM, check for the ASID (Application Server ID), Port, and SSL-enabled flag (IIOP/SIOP) on the Modify IMG screen. Change these settings as needed. • Verify that the IIOP transport parameter is enabled on the chassis by entering the show configuration | grep iiop-transport command. If it is not, enable using the instructions found in the System Administration and Configuration Guide. • Check if the SSL is enabled and/or enforced on the WEM. If the SSL is enabled, disable the IIOP transport on the chassis and set the value of IMG Port for the chassis such that it is identical to the SIOP port parameter configured on the chassis.

‌

Problem:	Received “Callbacks between server and client are not working. Screen cannot be invoked.” message.
Possible Cause(s):	• A firewall is prohibiting communication between the client and the server. • The network connection between the server and client machine might be slow.
Action(s):	• Edit the img.html file (located in the <ems_dir>/client directory (by default) to use fix ports and open the required ports in the firewall. This requires the configuration of the "FIXED_PORT", "FIXED_PORT_RANGE_START" and "FIXED_PORT_RANGE_END". • Verify that all the ports between FIXED_PORT_RANGE_START and FIXED_PORT_RANGE_END (both inclusive) are open on the firewall. If they are not, they should be opened. • The user can try performing a "ping" or “traceroute” from the server machine for the client's IP address and check for any packet-loss or network delay and contact the network administrator if required.

‌

Issues Related to Bulk Statistics

Problem:	WEM server not receiving bulkstats files.
Possible Cause(s):	• No FTP server is running on the server. • User specified in bulkstats configuration screen to FTP files from chassis to WEM does not exist. • Invalid FTP user password is being used to access the chassis. • No “destination” path for bulkstats files is configured. • Inadequate user privileges for bulkstatistics storage directory. • Sun Solaris WEM Servers only: Solaris operating system patches may need to be updated. • Poor quality connection between chassis and WEM.
Action(s):	• Verify that the FTP server process is running on the server by issuing the ps -ef | grep in.ftpd command. If it is not, start it. • FTP user needs to be created by Administrator. • Check the username and password used to ftp the bulkstats data from the chassis to Web Element Management server. Compare the “FTPUserName” and “FTPPassword” parameters in the nms.cfg file located in the <ems_dir>/server/etc directory by default to the names of administrative users with FTP privileges on the chassis. • Verify that the “destination” directory is configured in the bsparser.cfg file located in the <ems_dir>/server/etc directory by default. • Verify that the FTP user has permissions to write to the configured storage directory. • Sun Solaris WEM Servers only: Verify that the latest Solaris operating system patches are installed. Refer to WEM Port and Hardware Information chapter of this guide for more information. • Ensure that the configuration the bulk statistics receiver on the managed system Executing the show bulkstats command on the chassis displays this information. The “Remote File Format” field should contain a valid directory on the WEM Server. (Also verify that this directory exists on the server.) The “Bulkstats Receivers” field should contain the IP address of the WEM Server. • Slow connections could affect the data transfer between the chassis and the WEM server. In this case, contact your system administrator.

‌

Problem:	XML files are not getting generated.
Possible Cause(s):	• The Bulkstats Server component is not running. • Invalid “sample-interval” parameter configuration on the system. • Bulkstatistics functionality was configured through the system’s command line interface rather than through the WEM. • XML file generation is disabled.
Action(s):	• Verify that the Bulkstats Server process is running by entering the ps -ef | grep bulkstatserver command. If it is not, execute the ./serv bulkstatserver start command from within the server directory (<ems_dir>/server by default). • Verify that the “sample-interval” parameter on the system is set to either “1” or “5”. The value can be determined by entering the show bulkstats command on the command line. • Make sure that the bulkstats schema configuration is done through WEM. • Make sure that “XMLDataEnable” parameter in the etc/bsserver.cfg file is set to “1” (enabled). If it is not, change the setting, save the file, and execute the ./serv bulkstatserver start command from within the server directory (<ems_dir>/server by default).

‌

Problem:	Received “No matching data found” error when fetching bulkstatistics reports.
Possible Cause(s):	• The bulk statistics may not be getting parsed properly. • Invalid filter criteria used for the fetch operation.
Action(s):	• Verify that the filter criteria entered is valid. • If you are querying server in another time zone, enter the timing filters according to your time zone only. • Check if the corresponding data is present in the bsdb. • Verify that the “sample-interval” parameter on the system is set to either “1” or “5”. The value can be determined by entering the show bulkstats command on the command line.

‌

Problem:	Bulkstats files are not getting parsed.
Possible Cause(s):	• The Bulkstatistics Parser component is not running. • Bulkstat configuration may have been done through the CLI.
Action(s):	• Verify that the Bulkstatistic Parser process is running by entering the ps -ef | grep bulkstatparser command. If it is not, execute the ./serv parserserver command from within the server directory (<ems_dir>/server by default). • Verify that the bulkstatistics format is compatible with the WEM. Refer to the bs.cfg file (located in the <ems_dir>/server/etc directory by default) for WEM bulkstatistic formatting.

‌

Problem:	Bulkstatserver process is not started.
Possible Cause(s):	The function of 'bulkstatserver' is only to generate the XML files. It is possible that the “Generate XML Files” option was not enabled during installation.
Action(s):	• For an existing installation, edit the “XMLDataEnable” parameter in the etc/bsserver.cfg file to be set to “1” (enabled). Once the setting is changed and the files is saved, execute the ./serv bulkstatserver start command from within the server directory (<ems_dir>/server by default). • For new installations, ensure that “Generate XML Files” option is selected during the installation process.

‌

Problem:	ASCII files are not getting generated.
Possible Cause(s):	Configuration may not be proper.
Action(s):	Check for the following combination of configurables in bsserver.cfg: 1) XMLFileType = 0 2) ASCIIFileGeneration = 1 3) Configure proper licenses in emslic.cfg To reflect changes in any of the configurable values, restart the bulkstatserver. Important: If XMLFileType is set to 1, it will generate XML files irrespective of the other two mentioned configurables.

‌

Problem:	File header in bulkstats file is not proper.
Possible Cause(s):	The header might have been configured through CLI.
Action(s):	Always configure the header format through WEM only; else it will affect the bulkstatparser functionality.

‌

Problem:	XML files are getting re-generated.
Possible Cause(s):	OverrideLastAccessFlag may be set to 1.
Action(s):	Check if OverrideLastAccessFlag in bsserver.cfg is enabled. If it is enabled, disable it and restart the bulkstatserver.

‌

Problem:	XML/ASCII files ftp operation failing frequently.
Possible Cause(s):	• Network problem • PoolSize in bsserver.cfg is not adequate
Action(s):	• Check the network health to see if there is too much delay or packets getting dropped. Contact the network administrator in case of issues. • If FTP is used, it is recommended that PoolSize should be 10. If SFTP is being used, Poolsize must be set to 3.

‌

Problem:	Schema counters are not fetched/plotted properly.
Possible Cause(s):	• Reconfiguration of schema is not done after upgrade. Refer to the Reconfiguration of Bulkstat Schemas section of this guide for more information. • Bulkstat parser is not running. • There is no data matching the filter criteria.
Action(s):	• After upgrade, reconfigure the bulkstat schema for reflecting the new schema changes, if any. • Start the bulkstatparser. • Check if the filter criteria is correct and ensure that the data is available for that period.

‌

Problem:	Counter names are visible in data files.
Possible Cause(s):	The counters are not supported on the system. There are two possibilities: • Counters removed from the boxer build. In this case, WEM cannot remove the counters from schemas as WEM maintains backward compatibility. This does not have any impact on the WEM functionality. • System build is older than WEM build (for example, StarOS build 7.0 and WEM build 8.0). This does not have any impact on the WEM functionality. Bulkstat parser will ignore the counter strings while parsing data files.
Action(s):	No action required. This is a normal behavior.

‌

Problem:	Issues with file 2-4 format.
Possible Cause(s):	WEM only supports 'file 1' format for configuring schemas - and only if it is configured through WEM. Problems related to other file formats are out of WEM scope.
Action(s):	No action required.

‌

Issues Pertaining to Configuration Backup

Problem:	Configuration files are not getting backed up.
Possible Cause(s):	• No FTP server is running on the server. • Invalid FTP username password being used to transfer files from chassis to WEM server. • Inadequate user privileges for configbackup storage directory. • Solaris WEM Servers only: Solaris operating system patches may need to be updated. • Poor quality connection between chassis and WEM.
Action(s):	• Verify that the FTP server process is running on the server by issuing the ps -ef | grep in.ftpd command. If it is not, start it. • Use the Configuration Backup screen to change the FTPUsername and FTPPassword. • Verify that the FTP user has permissions to write to the configured storage directory. • Solaris WEM servers only: Verify that the latest Solaris operating system patches are installed. Refer to WEM Port and Hardware Information chapter of this guide for more information. • Slow connections could affect the data transfer between the chassis and the WEM server. In this case, contact your system administrator.

‌

Issues Pertaining to Alarms

Problem:	The WEM is not receiving alarms.
Possible Cause(s):	• Invalid configuration of SNMP target parameters on the chassis. • An alternate application is receiving the alarms using the same port.
Action(s):	• Verify that the SNMP target IP address and port number configured on the chassis match that of the WEM server. The SNMP target configuration on the chassis can be determined by entering the show snmp transports command. Check this information against the WEM server IP address (“ServerIpAddress”, specified in the nms.cfg file) and the SNMP port number (“SnmpTrapPort”, specified in the fm.cfg file) parameters. (Both of these files are located in the <ems_dir>/server/etc directory by default.) • Ensure no other application is running that receives the alarms on the port. Port status on the WEM server can be checked by executing the netstat -a command.

‌

Problem:	Not able to receive Mail Notifications
Possible Cause(s):	• The E-mail server is not running. • Invalid Mail identification information configured within the WEM for notifications.
Action(s):	• Verify that the E-mail parameters are properly configured in the fm.cfg file (located in the <ems_dir>/server/etc directory by default). • Verify that the E-mail server is running. • Verify that the E-mail information configured in the Alarm Configuration dialog of the WEM is correct.

‌

Problem:	Script execution on receiving ALARM fails
Possible Cause(s):	• The WEM Script Server is not running. • The script file is missing. • The script file does not have proper executable permissions.
Action(s):	• Verify that the Script Server is running by entering the ps -ef | grep scriptsrv command. If it is not, execute the ./serv scriptserver command from within the server directory (<ems_dir>/server by default). • Verify that the script file is located in the <ems_dir>/server/scripts directory (this is the default directory). If it is not, copy the script to that location. • Verify that the script can be executed by entering the ls -al command from within the directory in which the script is located.

‌

Issues Pertaining to the Process Monitor (PSMON)

Problem:	WEM processes are not restarted after a crash.
Possible Cause(s):	• PSMON is not running • Invalid PSMON configuration • The PSMON may have given up after performing multiple retries in a specific duration
Action(s):	• Verify that PSMON is running by entering the ps -ef | grep psmon command. If it is not, start it using the instructions located in WEM Process Monitor chapter of this guide. • Verify that PSMON is configured with the proper entries to start WEM processes. These entries may not be available if they were not selected for monitoring during the installation process. Refer to the instructions located in WEM Process Monitor chapter of this guide for information on PSMON configuration. • The PSMON tries to restart the processes for "numretry" time within a duration of "tmintval" (refer to etc/psmon.cfg) per process. If the process still doesn't start, PSMON no longer monitors this process. Please check the <ems_dir>/log/watchdog.log for details. Try restarting the process using the serv script.

‌

Issues Pertaining to Starting and Stopping EMS Processes

Problem:	When starting or stopping a WEM process, the user receives the error message ld.so.1: httpd: fatal: libgcc_s.so.1: open failed: No such file or directory.
Possible Cause(s):	Solaris WEM Servers: The configure runtime linking environment (crle) Default Library Path (ELF) on the Solaris server does not contain the EMS library path. Red Hat Enterprise Linux WEM Servers: The configure dynamic linker run time bindings (ldconfig) Default Library Path on the RHEL server does not contain the EMS library path.
Action(s):	Solaris WEM Server: • Log into the WEM Server as root and enter the crle command to view the current Default Library Path path. Here is an example where the crle path does not contain the EMS library path: # crleConfiguration file [version 4]: /var/ld/ld.config Default Library Path (ELF):/lib:/usr/lib:/opt/gss/lib:/opt/postgres/lib:/users/gss/lib:/users/postgres/lib:/users/usr/lib Trusted Directories (ELF):/lib/secure:/usr/lib/secure(system default) Command line: crle -c /var/ld/ld.config -l /lib:/usr/lib:/opt/gss/lib:/opt/postgres/lib:/users/gss/lib:/users/postgres/lib:/users/usr/lib • If the crle Default Library Path does not contain the EMS library path, use the following crle command to update it: # crle –u –l $EMS_INSTALL_PATH/server/lib
	Red Hat Enterprise Linux WEM Server: • Log into the WEM Server as root and verify that the ems.conf file is present in the /<ems_dir> /etc/ld.so.conf.d/ directory. To view the exsiting ems.conf file, enter the following commands: # cd /etc/ld.so.conf.d # cat ems.conf #This file is used to setup the EMS runtime linking environment • If the ems.conf file is not present, enter the following commands to create it: # cd /<ems_dir>/ems/server/lib # cd /etc/ld.so.conf.d # echo "$EMS_INSTALL_PATH/server/lib" > ems.conf • Verify that the /etc/ld.so.conf file contains the ld.so.conf.d/*.conf entry. If the entry is not present, add it to the /etc/ld.so.conf file. • Enter the following command to apply any changes made: ldconfig

‌

Issues Pertaining to Java

Problem:	The following message is displayed when attempting to access WEM dialogs containing tables after changing the look and feel option to Windows: “java.lang.NullPointerException”.
Possible Cause(s):	This may be an issue related to JRE 1.4.x.
Action(s):	If JRE 1.4.x is currently installed, uninstall it and then install JRE 1.5.

‌

Problem	The WEM client hangs when the user tries to load the configuration from the client file system. This is typical with older WEM builds
Possible Cause(s)	There is a bug in JRE 1.6.0_24 and above where JFileChooser class checks for the “modifyThreadGroup" permission which may not exist in the current .java.policy file.
Action(s)	Update the.java.policy file on any client machine that uses JRE 1.6.0_24 and above with the following line: permission java.lang.RuntimePermission “modifyThreadGroup."

‌

Problem	An error message displays to update the.java.policy file when the user invokes the WEM url. This is typical with newer WEM builds.
Possible Cause(s)	There is a bug in JRE 1.6.0_24 and above, where JFileChooser class checks for the “modifyThreadGroup" permission, which may not exist in the current .java.policy file.
Action(s)	Download the .java.policy file again from the options located on the WEM splash screen.

‌

Problem:	The WEM Process Monitoring dialog shows “Could not connect to server. Screen will not be invoked”.
Possible Cause(s):	This may be an issue related to JRE 1.4.x.
Action(s):	If JRE 1.4.x is currently installed, uninstall it and then install JRE 1.5.

‌

Problem:	After enabling audio support from the Alarm Management menu, the browser window crashes with the following error message: An unexpected exception has been detected in native code outside the VM. Unexpected Signal: EXCEPTION_ACCESS_VIOLATION (0xc0000005) occurred at PC=0x7131B60 Function=[Unknown.] Library=C:\Program Files\Java\j2re1.4.2_04\bin\jsound.dll In addition, the stack trace output shows: Current Java thread: at com.sun.media.sound.MixerThread.runNative(Native Method) at com.sun.media.sound.MixerThread.run(Unknown Source)
Possible Cause(s):	This may be an issue related to 1.4.x.
Action(s):	If 1.4.x is currently installed, uninstall it and then, install JRE 1.5.

‌

Issues Pertaining to WEM Upgrade

Problem:	The migrate script is not working if the postgres server is running on a port other than the default port.
Possible Cause(s):	The postgres communication port parameter is not passed to the postgres application call through the migrate script.
Action(s):	A script will prompt you to provide inputs for the postgres communication port at the time of data restoration (not at backup time) along with other input parameters (backup dir, WEM application path, postgres admin, password).

‌

Problem:	Map could not be fetched after manual upgrade from version 'A' to version 'B'.
Possible Cause(s):	While manually upgrading from version 'A' to version 'B', you may have used the original migrate script, i.e., the migrate script from version 'A' (since it is already available on the system) to restore the database.
Action(s):	While manually upgrading from version 'A' to version 'B', if you use the migrate script from version 'A', the database will be backed up. However, to restore the database, you have to use the migrate script from version 'B', i.e., the script that is packaged with the version that you are upgrading to. You will get the latest migrate scripts for restoring the database after unpacking the lastest WEM install.xxx.tar.gz bundle. The reason for doing this is that the recent scripts are likely to be upgraded from previous versions.

‌

Capturing WEM Client Logs

In the event that an issue exists that could not be solved using the information provided previously in this chapter, you may need to capture client logs for debugging purposes. This section provides information on how to utilize logging for WEM clients.

Step 1

Launch the web browser on the client machine.

Step 2

Open the Java Console. Assuming that the client is a Microsoft Windows-based machine, this can be done using the following instructions:

Step a

Right-mouse click on the Java(TM) 2 Platform icon in the status area (Windows System Tray).

Step b

Select Open Console from the menu.

Step 3

Open the URL of the Web Element Management Server in the web browser, replacing img.html with imgdebug.html.

Step 4

Login to the WEM and perform the operations causing the issues.

The Java Console contains log messages that could be used for debugging the issue.

Capturing WEM Server Logs using Script

In the event additional troubleshooting assistance is required, debugging information can be collected using a script called getSupportDetails.pl. This script collects different log files and captures the output of certain system commands that aid in troubleshooting issues. This script is packaged with the WEM Server in the <EMS_INSTALL_DIR>/tools/supportdetails/ directory.

This script refers to an XML file to get the list of logs. This XML resides in the same directory as the script. Once executed, the script retrieves the contents of logs, files, folders, and output of certain commands and prepares a zipped file (/tmp/log/emssupportDetails.tar.gz), by default it is placed in /tmp/log directory.

 

Requirements:

Perl 5.8.5 and above is required for running the script. This is packaged with the WEM Server.

Apart from standard Perl modules (which are included in default installation of Perl), some additional modules are required for running the script. The list is as follows:

•	expat version 1.95.8

•	expat version 1.95.8

•	XML:Parser version 2.34

•	XML-Parser-EasyTree

•	Devel-CoreStack version 1.3

These modules are installed by default by the WEM application. Please ensure that the above mentioned modules are installed when using a different installation of Perl.

To run the script, go to the path where the script is present and enter:

./getSupportDetails.pl [--level=...] [--xmlfile=...] [--outputDir=] [--help]

--level	Specifies the level of debug to run. It can have a maximum of 4 levels. The level 4 provides the most detailed information. Refer to README.txt file for more information. Default: 1
--xmlfile	Specifies the xml file name to be used for collecting the log. Default: getSupportDetails.xml
--outputDir	Specifies the output directory for the emssupportDetails.tar.gz file if different from the default output directory (/tmp/).
--help	Display this information.

‌

For example:

./getSupportDetails --level=4 --xmlfile=/tmp/something.xml --outputDir=/mywemlogscripts

WEM IP Address Change Procedure

In the event the customer’s network evolves, the IP address of the WEM server might be required to change from the existing one. In order to change the WEM server IP address, use the following defined IP planning process:

Step 1

Stop the EMS server using the following command:

./serv stop

Step 2

Change the interface IP address using the ifconfig command. For example:

ifconfig bge0 192.168.1.1 netmask 255.255.255.0 up

Step 3

Change the new IP address in the nms.cfg file of the WEM configuration. For this, use the vi editor as follows:

vi nms.cfg

Replace the IP address with the new IP address in the modify serverIpAddress field. For example:

ServerIpAddress = 192.168.1.1

Save the file after making the appropriate changes.

Step 4

Modify the host file of the WEM server using following commands:

# cat /etc/hosts

<IP address> localhost

<new_IP_address> solaris_hostname

#

Step 5

Start the WEM server after these settings changes using the following command:

./serv start

Important: /etc/netmasks needs to be modified if the user is subnetting existing address and subsequently using a different network mask than the default one. If the netmask being used for a given IP address is a default one, then there is no need to modify this file.

 

 

Cisco Systems Inc.

Tel: 408-526-4000

Fax: 408-527-0883